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Abstract 

In this paper, it is shown that the solutions of general differentiable constrained optimization problems can 
be viewed as asymptotic solutions to sets of Ordinary Differential Equations (ODEs). The construction of 
the ODE associated to the optimization problem is based on an exact penalty formulation in which the 
weighting parameter dynamics is coordinated with that of the decision variable so that there is no need to 
solve a sequence of optimization problems, instead, a single ODE has to be solved using available efficient 
methods. Examples are given in order to illustrate the results. This includes a novel systematic approach 
to solve combinatoric optimization problems as well as fast computation of a class of optimization problems 
using analogic circuits leading to fast, parallel and highly scalable solutions. 


1 Introduction 

Consider the following optimization problem with inequality constraints: 

min/(a;) under Q(a:) < 0 fG {!,...,nd (1) 

where / and c^’s are scalar functions of the decision variable x G M". Let / be the exact penalty induced 
function defined by: 

f{x,p) = f{x) + p-i^{x) (2) 


where ipix) is given by: 


ric 

ip{x) := V[maxjO,C;(3:)}]™ ; mGN (3) 

i=l 

A wide class of algorithms intends to solve Q by solving a sequence of unconstrained optimization problems 
of the form 


min/(a:,pfc) (4) 

for a varying (generally increasing) values of the weighting coefficient pk- For each problem in the sequence, 
only X is searched for while pk is kept constant mm- The series of unconstrained problems are sometimes 
replaced a series of problems with by box constraints as descent method with projection are easy to perform. 

The increase of p is generally defined by Pk+i ^ r x pk with r > 1. The sequence of precision parameter 
uik (to which the intermediate problems have to be solved) is also made such that w/j —>■ 0 in order to avoid solv¬ 
ing with a uselessly high precision intermediate problems. It is then obvious that the efficiency of the resulting 
algorithms is tightly related to the choice of r and the intermediate precision sequence {uJk}k since small values 
of r > 1 leads to unnecessarily high number of intermediate problems while a too high values of r > 1 leads to 
stiff problems that may lead to slow convergence (because this makes the solution of the box-constraint sub¬ 
problem harder m) beside the fact that it breaks the continuation argument that underlies the whole scheme. 
The choice of ujk corresponds to similar trade-offs that need to be carefully handled. Such issues are extensively 
studied in [T] leading to non necessarily monotonic behavior of pk when solving the sequence of intermediate 
box constrained modified Lagrangian problems in order to avoid high number of iterations that result when pk 
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is unnecessarily high. This recent study [T] shows at least that monitoring the dynamic evolution of pk is not a 
trivial issue. 

In this paper, it is shown that simultaneous dynamics of x and p can be defined through a differential equation 
of the form: 


x = Fi{x,p) ; p = F 2 {x,p) (5) 

such that solving ([^ gives trajectories that asymptotically converge towards the set of solutions of Q. 

Note that by doing so, the present paper does not propose a specific alternative algorithm to solve the NLP 
problem Q. Rather, it enables all the efficient algorithms that are available through the huge literature on 
ODE integration to become candidate algorithms for Q . Moreover all the computational background regarding 
many issues (such as parametric sensitivity [7], parallel computing [13], precision monitoring to cite but few 
items) become available for the constrained optimization problem paradigm. As such, the result of the present 
paper can be viewed as a starting point for future investigation. More interestingly, it is shown briefly in this 
paper that expressing the fact that solving Q can be done by integrating ODEs enables (for some specific 
problems) to built analogic circuits that can achieve the task in fast, parallel and massively scalable way. 

This paper is organized as follows: first section |1.1| gives the definitions and notation used throughout the 
paper. Section [^states the working assumptions that are needed to derive the main results of the paper. These 
results are stated and proved in section]^ while section]^ gives some examples of application of the paper results. 
Finally the paper ends with sectionj^that summarizes the contribution and gives hints for further investigations. 

1.1 Definition Notation 

Throughout the paper, the following notation is used. The n-dimensional vectors fxix), 'ipxix) and fx{x,p) 
denote the gradients of /, ip and / w.r.t x. The scalar function g{x,p) denotes the euclidian norm of fx at 
{x,p), namely: 

9{x,p) ■= \\fx{x,p)\\ ( 6 ) 

For a given weighting coefficient p > 0, the set of stationary points of f(-,p) is denoted by iSp, namely: 

Sp := {a; e K” I g{x, p) = O} (7) 

For any x G K", the notation d{x,p) refer to the distance between x and the set Sp, namely: 

d(x, p) := min ||a; — z\\ (8) 

zeSp 

The set of admissible values of x is denoted by: 

A := {x S K" I ip{x) = 0} (9) 

2 Working Assumptions 

The first assumption states that the optimization problem is well posed in the sense that either the original 
cost f{x) is already lower bounded or the constraints are such that the weighted cost / is lower bounded: 


Assumption 1 [Well posedness] For any p > 0, there is a lower bound /min(p) such that f{x,p) > fmin{p) 
for all X G K". 


Note that in the framework of the present paper, it is not assumed that the functions involved are convex. 
This means that the set Sp may not be a singleton {x*(p)}, it is assumed that the norm of the gradient of the 
weighted cost /(•, p) away from Sp can be bounded below by the distance to the set Sp through some coefficient 
kc- This leads to the following generalization of the strong convexity assumption: 
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Assumption 2 [(5'p)-Strong Convexity] There is a constant kc > 0 such that the following inequality holds: 


g{x,p) > kcX d{x,p) 


( 10 ) 


for all X G K". 


Note that contrary to the classical strong convexity assumption that involves two arbitrary points xi and X 2 , 
the inequality (101 involves the distance from an arbitrary x to those points lying inside the set of stationary 
points Sp. 


The Next assumption describes a generalized Lypschitz-like assumption on the constraints and the way they 
are used to construct the exact penalty term 'rp{x). 


Assumption 3 [Growth rate of fj] There is a polynomial P of degree G N with P{0) — 0 that satisfies 
the following inequality 

\'>P{X2) -tf{xi)\< P{\\X2- Xi\\)] (11) 


for all X G K". 


Note that the use of the polynomial P of the form: 

Tlijj 

P(d):=^a,d* (12) 

i=l 

accounts for the possibility to use different penalty exponents m in the definition of the constraint penalty term 
in ([^ and the fact that the bounding function may involve lower powers for small distances d and higher powers 
far from the set A. 

The following assumption is needed to guarantee the existence of solutions to the ODE built up with the 
functions fx and ip: 


Assumption 4 [Locally-Lypschitz maps] For all finite p> 0 the maps fxi‘,p), '</’(■) ipx{‘) are locally 
Lypschitz. 


Note that this last assumption expresses local requirement while (10) and 0 are required to hold for any x. 


The last assumption concerns the relevance of the use of the penalty method to solve ([^. It states that 
when the penalty coefficient p goes to infinity, the possible stationary points for the weighted cost converge 
towards the admissible set A: 


Assumption 5 [Relevance of the penalty approach] 


lim sup ip{x) 

p-s-oo L xeSp 


= 0 


(13) 


This assumption is almost implicitly required in any penalty-based approach to solve the constrained optimiza¬ 
tion problem ([^. It can obviously be replaced by some more apparently trivial assumptions that can be used 
to prove ([T^. The short form is preferred here for the sake of clarity. 


3 Main Results 

The main result of the paper can be stated in the following proposition: 
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Proposition 1 [Main Result] Assume that some {X,q) G x N is chosen. Consider the following system 
of differential equations: 


X 

P 


■ 9{x,p)y 

■ (z-l)! J 

7 X 'ip(x) 


X fx(x,p) 


(14) 

(15) 


if the following conditions hold: 

1. Assumptions^Tj^ are satisfied 

2. q> n^jj [see 


then for any A > 0, there is a sufficiently small 7 > 0 such that any asymptotic solution of satisfies 

the KKT necessary conditions of optimality for the constrained optimization problem 0 . ^ 


Proof. Let us compute the derivative of the weighted cost f{x,p): 


dl 

dt 


2=1 


[>^- 9 {x,p)y ^ 

{i-iy. 

(A ■g(a:,p))^ i2 

A-(*-l)! J 


2 

X g‘^{x,p) 


(16) 


Let Xa{p) G Sp be the closest point to x that lies inside the stationary set Sp. According to (11) of Assumption 
one can write: 


y{x) < ifixsip)) + p{\\ 

X - Xs(p)ll) 

(17) 

< y{Xs{p)) +P{d{x,p)) 

(18) 

Now by virtue of Assumption]^ tp^Xsip)) satisfies the following asymptotic property: 


y{xs{p)) = o{i/p) 


(19) 

Therefore, (18) becomes [using ([T^] : 



y{x) < P{d{x,p)) + 0(l/p) 

( 20 ) 

< \^aid\x,p) 

+ 0 (l/p) 

( 21 ) 


Z =1 


On the other hand, the sum in the r.h.s of (IT^ satisfies [because of (|l0|] the following inequality: 


[S 


{^■9{x,p)y 


^ A.(*-l)! 


> 


[S 

2=1 


{X-kc- d{x,p)y 


A •(*-!)! 


( 22 ) 


Now using (21) an d ([2^ in (16) enables to write [dropping all the terms with indices higher than np < q in the 
summing term of (|16[)] and using the identity 2/1 — gl = ( 2/1 + g 2 )(g 2 — yi): 


dl< 

dt 


/3+d* + 0(1/p)] ■ [J2 pyd^ + 0{l/p) 


2=1 


2=1 


(23) 


where d* := d^{x,p) while and /3j are given by: 


Pt = VtQ;* + 


= V7Q;i - 


{xk^y 


A •(*-!)! 

{Xk,y 

A •(*-!)! 


(24) 

(25) 


4 
























and taking 7 sufficiently small so as to satisfy the following inequality: 

{xk,y 


Tl-ijj 

< min 


the following inequalities hold for 13^ and (3^ : 




{Xkc 


2a,(A-(z-!)!)_ 

{\k,y 


A •(*-!)! 


yy<- 


2A-a,((f- 1)!) 


(26) 


(27) 


With these inequalities, inequality (23) implies: 

{xk,y 


dl 

dt 


< - 


Tltp 

E (A •(*-!)!) 

TLijj 


[E 


(Xkc 


^2(A-(z-l)!) 


dyx,p) + 0(1/p) 


d\x,p) +0{l/p) 


(28) 


Let us now show that inequality (28) implies that limt_>oo = 0. Indeed, if this was not the case, then by 

the very definition of the dynamic on p |^e it comes that p goes to infinity. This together with ( [2^ and 
the lower boundedness of / [Assumption m implies that limt_>oo d{x,p)=0 {x converges to the set Sp). But this 
implies by (13) of assumption ^that yi^x) converges to 0 which contradicts the assumption. Now since ip{x) con¬ 


verges to 0 , the inequality (llfi) together with the lower boundedness of / implies also that g{x, p) converges to 0 . 


By now it has been shown that provided that 7 is sufficiently small to satisfy (26), the trajectory of {x^p) 
converges to the following set 


|(x,p) I 5 (a:,p) = 0 and ■!/’(a:) = o| 


(29) 

It remains to prove that if (a;, p) belongs to the set defined by ( p^ , then x satisfies the KKT necessary conditions 
of optimality. Remember that these conditions require the existence of a vector p G such that the following 
conditions hold: 


(30) 


f)c ■ 

/x(^)+Er=iM. 7 ^(^) = o 



Ci{x) < 0 

(V^G{ 1 ,.. 

■,nc}) 

di>0 

(Vi G 

■,nc}) 

Pi X Ci(x) = 0 

(VzG{l,.. 

■,nc}) 


But g(x,p) = 0 can be explicitly written as follows: 

Tic O 

fx{x) + py. [max{0,Ci(a;)}]'"“^ x -^[x) 

i=l 

which obviously shows that by taking p such that: 

dl ■= 


0 if Ci{x) < 0 

p X m X [ci{x)Y^~^ if Ci\x) > 0 


(31) 


(32) 


the first KKT condition is satisfied by construction. The second condition {ci{x) < 0) results from y{x) = 0. 
The third and the fifth conditions {pi > 0 and pi ■ Ci{x) = 0) result from (32). This ends the proof. □ 


Note that if the summation in ( |14[ ) is performed with an infinite number of terms, the following corollary 
can be obtained: 


Corollary 3.1 Assume that some X > 0 is chosen. Consider the following system of differential equation: 

X = -exp[X-g{x,p)]x f^{x,p) (33) 


p = 7 X y{x) 


(34) 


If Assumptions 7][5 hold then for sufficiently small 7 > 0, any asymptotic solution of l[3djf-{ 34) satisfies the 
necessary conditions of optimality for the constrained optimization problem 
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Note that the result of Proposition[T]holds for any initial condition that can be used to initialize the trajectory of 
(14)-(151. The price to obtain such a global result lies in the use of the g-term summation that premultiplies the 

The next proposition gives a weaker result that can nevertheless be preferable 


gradient term —fx{x,p) in ( |l4[ ) 
in some circumstances. In this weaker result, the convenient sufficiently small 7 would depend on the initial 
values of x and p. 


Proposition 2 [A Simpler Weaker Result] Assume that some A > 0 js chosen. Consider the following 
system of differential equations: 


X = -fx(x,p) (35) 

p = 7 X ipix) (36) 


If the following conditions hold: 

1. Assum‘ptions\T^are satisfied, 

2. /(•) is proper (that is lim|| 2 ,||_,.oo fix) = 00 ) 

then for any initialization (xo,Po), there is sufficiently small 7 > 0 such that the resulting asymptotic solution 
of (35)-(36) satisfies the KKT necessary conditions of optimality for the optimization problem 0. ^ 


Proof. Note that the result can be obtained if one can show that everything behaves as if np = 1 holds in 
(11). Indeed, in this case g = np = 1 can be used and (141 is equivalent to (35). This can be done using classical 
arguments that are typically used to derive semi-global results. More precisely, given the initial state (xo,po), 
dehne the following level set in K": 


y(xo,Po) '■= |a: I f(x) < 2/(a;o,po)| 


(37) 


to which the initial value xq obviously belongs [because f(xo) < fixo,p) for all p\. Note that since / is proper 
by assumption, the set V(xo,po) is a compact set. Consequently, there is some sufficiently high di such that 
the following inequality holds for all ixi,X 2 ) G V{xo,po). 


\\'i(ix 2 ) - f’ixi)\\ < Pi\\x 2 - a::i||) < Cii\\x 2 - xi\ 


(38) 


this means that as far as the trajectory remains in V{xo, pf), the result of Propositioncan be used with np = 1 
therefore, there exists sufficiently small 7 such that the dynamics defined by ( |I4[ ) with g = I [which is the same 
as (35)] decreases the value of f{x,p). But this guarantees that the trajectory of x remains in V(a;o,/9o)- This 
implies that the inequality (38) remains true and the result obviously follows. □ 


Note that such finite di > 0 exists as long as the last inequality in (38) is required only on the compact 
set Vixo, pq). The latter is defined in terms of the initial paire {xq, po). This is why the value of di does depend 
on the initialization and may not exit globally. 


3.1 General Comments 

Before getting to the examples section, it is worth mentioning that the results of the present section build a 
theoretical bridge between NLP and ODE algorithms in a rather systematic way and for a large class of prob¬ 
lems. However, it must be underlined that although the following examples show rather efficient computational 
results, the integration of the resulting ODE may not be the more efficient way to solve the underlying opti¬ 
mization problems. This is because integration schemes try to reproduce high precision solution over the whole 
trajectories while from the NLP solution point of view, only the asymptotic trajectory matters. 

To this respect, the results of the present section can be used to derive gradient-based algorithms (fast gradient 
for instance muni) using the r.h.s of the ODE as extended gradient in the extended space of {x, p) with / as 
cost function. By doing so, even certification results similar to the one proposed in m can be extended from 
the case where only saturations on the control input is used to the more general case of affine constraints on the 
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i){x) f{x)/f{x°P*) f{x)/fix°P^) 



Figure 1: Example 1. Trajectories of ipi^) f{x) for 50 different randomly generated problems. Note that 
the constraints are asymptotically satisfied 0) and that the resulting costs converge towards the exact 

optimal value f{x°P*) (since ///°^* —>■ 1). For all trials, the initial conditions (0,0) is used. Initial values of ip 
shows initial strong violation of the constraints. 


state. This being said, no such efficiency-oriented development is done here focusing on the main theoretical 
contribution of the paper. 

On the other hand, another consequence of the theoretical result of the present section is the possibility to 

built electronic circuits that realize analogic ultra-fast integration of the ODE for a class of NLPs. This is 

briefly discussed in section |4.2[ 

4 Illustrative Examples 

4.1 Example 1: QP problems 

As a first examples let us consider the use of the ODE framework described in Proposition!^ to solve Quadratic 
Programming (QP) problem with inequality constraints. This leads to the following instantiation of the cost 
function f{x) and the constraints Ci{x): 

f{x) = ^x^Hx + F^x (39) 

Cj(x) = AiX-B^ ; i = l,...,nc (40) 

where H G M"^”, F G Ai G and Bi G M. Now using m = 2 to define the constraints-related 

weighting term: 


Tie 

ip{x) = [maxjO, AiX — i?i}] 

i=l 


(41) 


gives = 2 [see 
<7 = 2): 


@ and @]. Consequently, following Proposition]^ the following ODE is defined (taking 


where 


X 

P 


-[^ + >'-\\fx{x,p)\\ xf^{x,p) 

TLc 

7 X ^ [max{0, AiX — Bi}'\ 


2=1 


Tie 

fx{x,p) = Hx + F + py^^ maxjO, AjX - BJ] x Af 
2 = 1 


(42) 

(43) 
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Evolution of Xj, z = 1,..., 15 



Figure 2: Example 1. Typical behavior of of the components of x on the trajectories of the ODE. Initial value 
(0,0) G K" X K is used. 


This ODE is then integrated using the Matlab ODElSs stiff solver to get the solution of the original QP 
problem defined by (39)-(40). Fifty randomly generated sets of matrices {H, F, A, B} are generated leading 


to 50 feasible QPs with n = 15 unknown and Uc = 20 constraints. The resulting ODEs (42)-(43) are defined 
with the parameters A = 10“"^ and 7 = 10“®. Figure shows the resulting trajectories of ip and the cost 
function normalized by the optimal cost value (computed using the standard Matlab QuadProg solver). All 
the trajectories are started from xq = 0 and p = 0. The Figure clearly shows that the trajectories converge 
to the solutions of the problems as the constraints are satisfied and the cost function values converge toward 
the optimal values for all the generated problems. Figure shows a typical behavior of the system’s trajectory 
starting from (0, 0) and converging towards the optimal values. The computation times shows a mean of 49 ms 
with a variance of 3 ms (Using Matlab on Mac PowerBook OSX, 2.8 GHz Intel Core i7 processor). 


4.2 Example 2: Analogic MFC Solvers 

Recall that Model Predictive Control (MPC) for linear time invariant systems of the form: 

i = A^ + Bu 

is based on the repetitive solution of a quadratic programming problem of the form: 

. rl - - 


mm -x^Hx + [fo + Fi^ 
kgR"L2 


(44) 


(45) 


under the constraint: 

AiX - [b° + B^^] < 0 ie{l,...,nc} (46) 

where x G M" is the parameter vector that defines the control trajectory over the prediction horizon, namely: 

u{k) 


= n • a; 


(47) 


yu{k + N-l)^ 

for some appropriately chosen parametrization matrix H G where riu is the dimension of the control 


input u. Note that the only difference between (39l-(40) and (45)-(46l is that the affine term in (45) and the 


r.h.s of the inequalities (46) depends on the state of the controlled system For more details on MPC design, 
the reader can refer to [S] 


































Now applying Corollary 3.1 to the QP defined by (45l-(46) with a sufficiently small 7 for all initial condi¬ 
tions of interest, it comes that the QP solution (for a given can be obtained by integrating the following set 
of ODEs: 


X 


P 


Hx + fo + 


Tla 

2 p^[max{0,yl,a; - b° - Bi^} 


■A{ 


Tic 

' ^ [niax{ 0, AiX — b° — 


i=l 


(48) 

(49) 


The idea is then to perform the integration through analogic circuits. The literature is very rich regarding the 
way transfer functions and more generally nonlinear differential relationships can be realized by analogic circuits 


(see mu and the references therein). Let us concentrate on the operations involved in ([4^-p^ to check that 
analogic realizations can be derived considering that x is represented by a vector of currents (voltage options is 
also possible [3] although it is not discussed here): 

• Constant current sources Note first of all that the constant terms /o and b'f corresponds to tunable source 
of currents. 


• State dependent current sources The state dependent terms and Bi^ are computed numerically and the 
results is assigned to another vector of current sources that remain constant during the integration step. 
This is the only numerical operation which determines the sampling rate of the resulting MPC controller. 

• Linear combination of currents. This concerns the terms Hx and AiX and can be realized for instance 
using unity gain cells as shown for instance in [^. 


• Current summation and substraction. This concerns the realization of the sums Fix + fo + F^ and 
AiX — b^ — Bi^ and can be viewed as a particular instantiation of the previous item and can therefore be 
realized using unity gain cells. 


Squaring signals. This is necessary to compute the summated terms in (49) and can be achieved for 
instance using the circuits proposed in HIS] or any later work containing more recent devices and archi¬ 
tectures. 


• Multiplieation by p. This operation can be realized through tunable gain or by using standard signal 
multipliers as the on proposed in [ 6 ]. 


Note that the time needed to analogically integrate (48)-(491 is the time necessary to fill the corresponding 
circuit’s capacitors. This time can be made extremely short (nano or even pico-seconds) if the problem is ap¬ 
propriately normalized so as to have its normalized solution components Xi scaled down so that they correspond 
to tiny capacitor voltages. 


Note that in the above presentation, the linear character of the controlled system plays no determinant role. 
Indeed, thanks to the possibility of signal multiplication, squaring and even the possibility to implement the 
square rooting of signals [1], a wide class of ODE’s that would be associated to the solution of a wide class of 
non quadratic constrained NLP can be analogically solved in extremely fast way. Moreover, the potential use 
of massively integrated circuit makes it possible to solve large scale problems in this way. 


Note finally that many of the above mentioned circuits can be realized using on-line assignable gains which 
makes it potentially possible to use the same circuits for many different problems. It remains however necessary 
to analyse the cost of such circuit design and realization which is beyond the scope of the present paper that 
studied the conceptual opportunities that are made possible by the ODE-related formulation of constrained 
optimization problems. 


4.3 Example 3: Solving Nonlinear Mixed-Integer Optimization Problems 

In this section, presentation is done for the special case where all the decision variables are binary. The 
case where some decision variable can be continuous can be obtained easily with extra notational complexity. 
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Consider the optimization problem given by: 


min f(x) under Ci(x) < 0 and Xi S {0,1} 


(50) 


for all i e {1 ,..., ric}- 

It is well known that this problem can be put in the standard form Q by transforming the binary constraints 
Xi S {0,1} into standard constraints of the form 


Xj — xj < Q — Xi < 0 and a:,- — 1 < 0 


(51) 


which yields a number of inequality constraints ric = Uc + 3n. Moreover, the integer that characterizes the 
growth of '0 [see (111 and (12) is given by = 2m where m is the exponent used in the definition ([^ of 0. 

Using m = 2 leads to the following definition of ip{x): 

ria 2 ” 

0 (a:) = E[ max{0, Ci(x)}j + j^maxjO, Xj — 


^[max{0,-a:jl + Uax{0, x, - 1} 


2=1 


2 = 1 


Now applying the result of Proposition suggests the solution of the combinatoric optimization problem can 
be done by integrating the following ODE: 


X = — 


4 

[E 

2=1 


(A • gix,p)) 


2-1 


(z-1)! 


X fx{x,p) 


p = 7 X ip(x) 


(52) 

(53) 


Now obviously the admissible set is not a convex set and the presence of local minima is very likely. The following 
algorithm can be used to visit such local minima successively. In this algorithm a successively modified cost 
function /^^^(•) where = / is initialized to the original cost and where s denotes the number of already 
visited local minima. The local minimum is found by integrating the ODE defined by the weighted function 
J^’^\x,p) and its corresponding norm of the gradient g^’^\x^p) := fx^\x,p), namely: 


4 

[S 


I = -| 

2=1 
p = j X 


(A-5r(^)(x,p))* 


(*-l)! 


X fi"\x,p) 


(54) 

(55) 


This is done starting from the initial condition (a::^®“^0O) and integrating the ODE until some stopping condi¬ 
tions on both 0 and g are satisfied. Then a term is added to the cost function which makes the current solution 
inappropriate. This can be done by first defining a neighbor vector to a;^®^ such that: 


= 1 and c{z^^^)<0 

The new cost function /(®+i) is now defined by: 

/(®+i)(x) := /(®)(x) + (1 + 2/(®)(z(®))) • exp(^||x - x(®)f) 

Now for sufficiently high p, this new cost function is such that x*-®^ is no more a local minimum since 

/(®+l)(x(®)) = /(®)(x(®)) -h + 1 > /(®H-2^"^) 


(56) 


therefore, incrementing s and firing the integration of the new resulting ODE (54|-(55) starting from the initial 
condition (x*'®0 0) leads to a necessarily different minimum and so on. 

The only assumption tha t is implicitly assumed is that there always exists a neighbor vector that is admis¬ 
sible in the sense of (561. If this is not satisfied, less close z^®^ can be searched provided that a deterministic 
generation process is defined. 
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5 Conclusion and Future Work 


In this paper, it is shown that the solution of optimization problems with inequality constraints can be obtained 
by solving appropriately defined ODEs. In these ODEs, simultaneous dynamics are given to the decision variable 
as well as to the weight associated to the exact penalty term on the constraint violation. One of the major 
impacts of this result lies in the possibility to design analogic circuits that can quickly and physically integrate 
the corresponding ODEs. Pushing this latter idea towards a concrete realization is the obvious follow up of 
the present work. Another direction is to use the result to derive a fast gradient algorithm together with 
its associated certification bounds regarding the number of iterations that would be necessary to achieve a 
prescribed level of precision following the steps of [H] while including affine constraints that are not considered 
in [12] . This was not possible precisely because when standard fast gradient is used, only projection on the 
box-like set can be done while guaranteeing the decrease of the cost function. The formulation proposed in the 
present paper provide generalization of this property to an extended monotonically decreasing cost function 
provided that the r.h.s of the ODE is used as an extended gradient. 
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